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1.0  INTRODUCTION 


-v  The  Fault  Tolerance/  Distributed  Computing  Laboratory  (FTDCL)  at 
Auburn  University  Is  a  facility  developed  under  the  sponsorship  of  the 
U.S.  Army  Ballistic  Missile  Defense  Advanced  Technology  Center 
(BMDATC)  for  the  purpose  of  conducting  experiments  to  test  and 
evaluate  various  candidate  fault  tolerance  techniques  for  use  in 
multiprocessor  computer  systems  which  may  be  employed  for  Ballistic 
Missile  Defense  (BMD).  The  FTDCL  is  unique  in  that  it's  structure  can 
be  modified  as  needed  to  support  various  hardware-oriented  approaches 
to  fault  tolerant  multiprocessor  design.  This  cannot  easily  be  done 
at  other  BMDATC  testbeds,  since  the  structures  of  these  systems  must 
remain  fixed  to  support  a  variety  of  users  on  a  daily  basis.  The 
FTOCL  facility  consists  of  a  tightly- coup  led  multiprocessor,  currently 
constructed  around  four  single-board  computers,  four  shareable  memory 
modules  and  an  experimental  "shuffl e/exchange w  interconnection 
network.  [1]  The  processors  in  this  system  work  in  parallel  on  various 
tasks  within  an  overall  problem,  cooperating  by  exchanging  and  sharing 
data  through  the  common  memory. 

^  1/79 V  vhs*  '  />» 

The  project  supported  by  the  U.S.  Army  Research  Office  under  contract 
0AAG29-82-G-0006 ,  entitled  "Hardware  Acquisition  for  the  Enhancement 
of  a  Fault  Tolerance/  Distributed  Computing  Laboratory",  is  summarized 


fn  this  report.  Under  this  project,  a  flexible  software  support 
environment  has  been  developed  for  the  FTDCL,  allowing  a  number  of 
tools  wh'lch  were  developed  to  support  BMDATC  work  In  the  Advanced 
Research  Center  (ARC)  In  Huntsville  to  be  utilized  for  experiment 
development  In  the  FTDCL.  This  support  environment  Is  hosted  on  a  VAX 

l 

11/780  computer  which  was  obtained  through  the  funds  provided  by  ARO 
under  this  project,  a  'donation  from  Intergraph  Corporation,  and 
additional  funds  provided  by  the  Engineering  Experiment  Station  at 

r 

Auburn  University.  Section  II  of  this  report  will  describe  the  FTDCL 
software  support  environment.  Section  III  will  briefly  describe  the 
major  development  work  that  has  taken  place  using  this  support,  and 
section  III  will  describe  several  related  projects  that  have  utilized 
the  system  purchased  under  this  project. 


II.  FTOCL  HARDWARE  UPGRADE 

A  block  diagram  of  the  FTDCL  system  Is  shown  In  Figure  1,  the  main 
components  being  the  four  processing  elements  (PE's),  the  four  shared 
memory  modules  (SM's),  the  Interconnection  network  (IN),  and  the  HOST 
computer  -  the  VAX  11/780.  The  HOST  consists  of  the  VAX  11/780 
processor,  2  Mb  of  memory,  900  Mb  of  disk  storage,  48  communication 
ports,  interactive  graphics  design  station,  and  8  pen  plotter.  Of  the 
communication  ports,  seven  are  interfaced  to  the  FTDCL  -  one  line  to 
each  of  the  four  PE's  and  one  to  each  of  the  three  terminals  used  for 


software  development  and  experiment  monitoring  and  control.  The 
remaining  ports  on  the  VAX  support  various  other  users  in  the 


Wr> 


Electrical  Engineering  Department  at  Auburn 


All  software  for  the  FTOCL  mult {processor  Is  entered  and  compiled  on 
the  HOST,  after  which  the' compiled  object  code  Is  down- loaded  Into  one 
or  more  PE's  via  their  serial  communication  lines.  In  addition  to  the 
standard  VAX  text  editors,  the  other  software  support  tools  utilized 
in  this  process  are  the  Process  Design  Language  (POL)  compiler 

a 

provided  by  the  BHD  ARC  [2],  the  Microbench  Z8000  Assembler,  and  a 

variety  of  1 ibrary  management  and  linking  tools  from  the  ARC  and 

Microbench,  as  we  I  1  as  a  number  of  tools  developed  at  Auburn.  Once 
,  code  has  been  compiled  Into  one  or  more  object  modules,  a  program  on 

I 

the  HOST  called  the  "Master  Operating  System"  (MOS)  is  used  to  allow  a 
user  to  Interface  with  the  FTDCL  multiprocessor  via  a  menu  of 
commands.  These  commands  allow  the  user  to  download  the  object  code  to 
one  or  more  PE's  in  the  multiprocessor,  run  and  interact  with  the 

experiment,  receive  results,  and  store  information  into  files  for 

later  analysis.  The  operation  of  the  MOS  program  is  more  fully 
described  in  [3]. 

On  the  FTOCL  multiprocessor  itself  is  a  three  layer  operating  system, 
as  shown  in  Figure  2.  The  Z8000  monitor  was  adapted  from  a  monitor 
program  supplied  by  Zilog  with  the  original  Z8000  processor  single 
board  computers  used  in  the  FTDCL.  The  monitor  provides  a  number  of 


communication  facilities  and  software  debugging  facilities  on  the 


board.  The  Kernel  Operating  System  (KOS),  described  in  [3],  provides 
the  run-time  support  for  PDL  programs  running  on  each  individual 
processor.  It  al lows  processes  to  be  scheduled  and  executed*  memory 
space  to  be  allocated  and  deallocated*  and  communications  to  take 
place  between  multiple  processes  on  the  system.  In  addition,  it  passes 
information  to  the  MOS  on  the  HOST  to  provide  experiment  feedback,  as 
well  as  receiving  information  from  the  HOST  for  experiment  control. 
The  KOS  i s  down  1 oaded  from  the  HOST  to  each  PE  pr I or  to  the 
downloading  of  any  application  programs  into  the  system.  The  third 
operating  system  layer  is  the  Global  Operating  System  (GOS),  described 
in  [4],  which  provides  overall  control  of  the  multiprocessor.  The  GOS 
is  responsible  for  scheduling  processes  to  be  assigned  to  PE's, 
maintaining  a  list  of  available  global  memory,  coordinating 
Interprocess  communication,  and  maintaining  a  number  of  error 
detection  and  recovery  features.  Error  detection  performed  by  the  GOS 
includes  diagnosis  of  IN  and  SM  faults,  maintenance  of  a  system-wide 
clock  in  conjunction  with  monitoring  process  deadlines,  and  other 
features.  The  GOS  has  been  made  flexible  enough  to  support  different 
scheduling  strategies  and  different  hardware  configurations  so  that  a 
variety  of  experiments  can  be  performed. 

Currently  in  the  FTDCL,  experiments  are  in  progress  which  will  be 
examining  the  operation  of  a  fault  tolerant  SM  module  and  various 
fault  tolerance  approaches  to  the  IN  design.  These  experiments  should 
provide  useful  information  in  the  design  and  developement  of  BMD  data 
processing  systems  in  the  near  future. 
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III.  RELATED  WORK 


The  VAX  11/780  system  purchased  under  this  project  has  provided 

I 

support  for  a  number  of  other  activities  related  to  Department  of 
Defense  research  activities  at  Auburn,  In  addition  to  the  software 
support  of  the  FTDCL  as  described  above.  This  section  of  the  report 
will  discuss  briefly  the  nature  of  some  of  these  related  activities. 

Computer-aided  design  (CAD)  is  extremely  critical  In  the  development 
of  large  and  very  large  scale  Integrated  circuit  (VLSI)  devices.  The 
Intergraph  VAX  11/780  system  with  Its  interactive  design  station  has 
been  used  to  develop  a  number  of -CAD  tools  for  VLSI  development, 
ranging  from  schematic  capture  to  system  and  circuit  simulation  at 
various  levels  of  detail,  chip  layout  for  mask  generation,  design  rule 
checking,  etc.  The  first  of  these  tools,  a  chip  layout  program,  was 
used  in  the  development  and  fabrication  of  a  mask  for  an  LSI  chip 
which  implements  a  switch  to  be  used  in  the  interconnection  network  of 
the  FTDCL,  described  above  and  in  [1,5].  This  chip,  when  fabricated 
and  tested,  will  allow  a  significant  reduction  in  the  components  of 
the  FTDCL,  and  will  facilitate  the  expansion  of  the  system  and  the 
Incorporation  of  redundancy  in  the  connection  network  cf  the  system 
for  fault  tolerance. 

CAD  tools  are  also  critical  for  the  study  and  evaluation  of  new 
computer  architectures  for  defense  and  aerospace  systems.  Several 


projects  at  Auburn,  sponsored  by  the  U.S.  Army  Missile  Command,  the 
Missile  intelligence  Agency,  and  NASA,  are  utilizing  the  VAX  11/780 
for  such' study  and  evaluation.  Activities  range  from  simulation  of 
systems  and  components  with  various  hardware  description  languages,  to 
the  development  of  expert  systems  for  use  In  defense  applications. 

i 

More  recently,  work  has  begun  In  the  study  of  power  systems  for 
space-based  defense  systems  at  Auburn  University's  Space  Power 
Institute.  The  VAX  11/780  described  above  will  be  used  for  a  variety 
activities  in  this  critical  study. 

IV.  SUMMARY 

The  hardware  purchased  with  funds  from  this  project  has  significantly 
upgraded  the  capabilities  of  the  .Fault  Tolerance/  Distributed 
Computing  Laboratory  at  Auburn  University,  supporting  studies 
involving  fault  tolerance  and  reliability  of  data  processing  systems 
for  BMD  appl ications.  The  VAX  11/780  has  provided  a  unique  software 
development  environment  to  support  experiments  on  the  FTDCL 
multiprocessor  system.  These  studies  have  and  will  continue  to 
provide  insights  to  8MDATC  regarding  techniques  for  the  design  of 
ultrareliable  systems,  identifying  the  various  cost,  performance,  and 
reliability  tradeoffs  of  candidate  fault  tolerant  design  techniques  as 
they  apply  to  BMD  applications. 

In  addition  to  support  of  the  FTDCL,  the  VAX  11/780  has  become  a  vital 
tool  In  a  variety  of  other  projects  sponsored  by  various  U.S.  Army  and 
other  government  agencies,  providing  support  for  CAD  activities. 
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system  simulation,  analysis,  and  in  general  providing  a  powerful 
computation  tool.  In  the  future,  this  system  will  continue  to  provide 
support'  to  these  and  other  defense  related  projects,  allowing  the 
Electrical  Engineering  Department  at  Auburn  to  maintain  the  high 
standards  it  has  set  for  Its  research  activities. 
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